Monitoring one or more articles on a support surface

ABSTRACT

Devices, systems and methods are provided for monitoring a plurality of articles that are arranged on a support surface according to an article arrangement modeled with a first processing device. One of the methods includes receiving image data generated from the modeled article arrangement. The image data is received by a second processing device, and is indicative of an image of the arranged articles on the support surface. The received image data is used to inspect the arranged articles.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 61/607,319 filed on Mar. 6, 2012, which is hereby incorporated byreference in its entirety.

BACKGROUND OF THE INVENTION 1. Technical Field

The present invention relates generally to article handling andmonitoring and, more particularly, to loading articles on one or moresupport surfaces with a mobile device and to monitoring the arrangedarticles before, during and/or after transportation of the loadedsupport surfaces.

2. Background Information

Transporting articles from one location to another is a common taskundertaken by a majority of businesses and various other enterprises,and is sometimes generally referred to as “material handling”. Themilitary and shipping services such as the U.S. Postal Service, FedEx®and the like, for example, utilize supply chains to pickup, transportand deliver various types of articles between different locationsworldwide. The term “logistics” may be used to describe flow managementof the articles and information for transporting, inventorying,warehousing, material handling and packaging of the articles within suchsupply chains.

Typical material handling and logistical tasks may include handlingarticles at their source, their destination and many points in-between.These tasks frequently involve packaging and/or repackaging a variety ofarticles into larger or smaller packages, pallets, containers and thelike for shipping across a distance. These tasks may also involveunpacking the articles at their destination for storage, use, deliveryor further transportation. Additionally, articles frequently are movedand stored at various places at various times within a particularbuilding or group of buildings, such as in a manufacturing operation,hospital, or warehouse as well as an aircraft, ship, cargo plane orother military environment.

At their source, order picking (also referred to as “palletizing”)typically is employed in a warehouse or other location where a number ofarticles are taken from a number of source locations, such as pallets,for example, and placed on a destination or shipping pallet or othershipping container or the like. Once filled, the pallet or container isthen taken to a loading dock or similar area where it may be packaged,such as with strapping, shrink wrap or the like, and loaded into atransport vehicle such as, for example, a truck, rail car, plane,helicopter, ship or other vehicle.

At their destination, the pallets or containers are removed from theirtransport vehicles. The articles may subsequently be removed from thepallets or containers and placed in another transport vehicle or anotherstorage location for further shipment, storage or use. This process andassociated tasks are routinely undertaken by businesses on a daily basisand are an integral part of providing cargo for private and military usein locations around the world and must be accomplished with a highdegree of accuracy to avoid additional costs and to sustain or increasecustomer satisfaction.

In many operations, order picking and pallet or container unloading isperformed using manual labor, which is not only time consuming and laborintensive, but is subject to errors and injury to workers particularlywith heavier loads. Additionally, workers typically use various types ofequipment to make such tasks easier, such as forklifts, pallet jacks andconveyor belts, for example, which not only require a substantial amountof capital for purchasing and maintaining such equipment but are limitedin the environments where they can be employed and create serious safetyhazards in the work environment.

Although automated systems have been developed, they typically are fixedinstallation types of systems that require a substantial amount ofinfrastructure installation and facility renovation and simply cannot beflexibly deployed in different warehouses, manufacturing facilities,planes, ships or other environments as needed. This requiredinfrastructure and renovation not only dramatically increases the cost,complexity and maintenance of such systems, but also limits theirapplication to large warehouses and operations with significant budgets.Additionally, if such systems go down for repair or maintenance theentire material handling operation is typically shut down if notsignificantly disrupted since those systems take up a substantial amountof floor space thereby inhibiting any backup systems from beingemployed, such as using forklifts or fork trucks, for example.

It would be desirable therefore to provide a readily deployable andnear-zero infrastructure solution to the above and other needs.

SUMMARY OF THE DISCLOSURE

Devices, systems and methods are provided for monitoring a plurality ofarticles arranged on a support surface according to an articlearrangement modeled with a first processing device. One of the methodsincludes receiving image data generated from the modeled articlearrangement. The image data is received by a second processing device,and is indicative of an image of the arranged articles on the supportsurface. The received image data is used to inspect the arrangedarticles.

BRIEF DESCRIPTION OF THE DRAWINGS

The following detailed description will be better understood when readin conjunction with the appended drawings, in which there is shown oneor more of the multiple embodiments of the present disclosure. It shouldbe understood, however, that the various embodiments of the presentdisclosure are not limited to the precise arrangements andinstrumentalities shown in the drawings.

FIG. 1 is an illustration of an electro-mechanical mobile device loadingone or more articles onto one or more support surfaces;

FIG. 2 is a system block diagram illustration of an electro-mechanicalmobile device;

FIG. 3 is an illustration of an autonomous mobile robot embodiment ofthe mobile device of FIG. 2;

FIG. 4 is an illustration of another autonomous mobile robot embodimentof the mobile device of FIG. 2;

FIG. 5 is a flow diagram of a method for loading articles on a supportsurface using the mobile device of FIGS. 1 and 2;

FIG. 6 is an illustration of an embodiment of a stable arrangement ofarticles on a support surface;

FIG. 7 is an illustration of a plurality of autonomous mobile robotsloading a plurality of support surfaces;

FIG. 8 is another illustration of the mobile robots of FIG. 7 loadingthe support surfaces;

FIGS. 9A-9F are illustrations of another embodiment of the mobile deviceof FIG. 2 arranging articles on a support surface;

FIG. 10A is an illustration of an article arrangement wrapped withpackaging material;

FIG. 10B is an illustration of the article arrangement of FIG. 10Asubsequent to being tampered with;

FIG. 11 is a block diagram illustration of a system for monitoringarticles in an article arrangement on a loaded support surface; and

FIG. 12 is a flow diagram of a method for monitoring articles in anarticle arrangement on a loaded support surface using the system of FIG.11.

DETAILED DESCRIPTION OF THE INVENTION

Autonomous devices, systems and/or methods may perform one or more taskswithout continuous outside control and/or intervention. An autonomousmobile robot, for example, may receive instructions to perform a certaintask at a certain location. The mobile robot may subsequently determineand perform the operation(s) necessary to move to the location and/orcomplete the task based on, for example, its current location,surrounding obstacles, its operating environment, the type of task to beperformed, etc. Such autonomous devices, systems and/or methods may alsoadapt to unknown, new and/or changing operating environments withoutadditional outside control and/or intervention.

Autonomous devices, systems and/or methods may be fully autonomous orsemi-autonomous. The term “fully autonomous” is used to describe adevice, system and/or method that may perform one or more tasks without,for example, any outside control and/or intervention. A fully autonomousmobile robot, for example, may perform a task without receivinginstructions (e.g., vectors, commands, etc.) from a human operatorduring performance of the task. The term “semi-autonomous” is used todescribe a device, system and/or method that may perform one or moretasks without, for example, continuous outside control. Asemi-autonomous mobile robot, for example, may perform a task utilizingone or more periodic instructions from a human operator that boundand/or qualify the performance of the task. The instructions, forexample, may provide an updated location of where the task is to beperformed, identify an unknown obstacle, control the scope of the task,control when the task should be performed, etc.

FIG. 1 illustrates an electro-mechanical mobile device 20 (e.g., anautonomous mobile robot) autonomously performing a task of loading oneor more articles 22, 24, 26 and 28 onto one or more support surfaces 30,32 and 34, where these articles may have different (or similar) shapes,sizes, weights, etc. Examples of articles include boxes, crates,containers, packages, barrels, canisters, luggage, chests, and amorphousbags, sacks and other such objects. Other examples of articles includeelectronic and/or mechanical devices and equipment, etc. Examples ofsupport surfaces include pallets, cargo bay floors, warehouse floors,truck beds, ship hold floors, etc. Other examples of support surfacesinclude interior surfaces of support containers, boxes and the like inwhich the articles may be placed and subsequently transported and/orstored. The present invention of course is not limited to any particulararticle and/or support surface type or configuration.

Briefly, the mobile device 20 may load one or more of the articles 22,24, 26 and 28 onto a first of the support surfaces (e.g., the transportpallet 30), for example, by gathering the articles from one or morerespective source pallets 36, 38, 40 and 42 and arranging the gatheredarticles onto the first support surface 30. Each of the source pallets36, 38, 40 and 42 is located at a respective article pickup location,and may support (e.g., hold) similar types of articles; e.g., the sourcepallet 36 supports the articles 28, the source pallet 38 supports thearticles 26, etc. Each of the support surfaces 30, 32 and 34 is locatedat a respective support surface location and may support one or moredifferent types of articles. The articles loaded onto the first supportsurface 30 may be selected based on, for example, a destination of thefirst support surface, intended (e.g., medical, culinary,warfare/combat, humanitarian aid, etc.) use of the articles, etc.Subsequent to the loading, the mobile device 20 or another device (e.g.,a fork lift, crane, conveyor, etc.) may move the loaded first supportsurface 30 onto a vehicle (e.g., a ship, train, truck, helicopter,airplane, submersible, spacecraft, etc.) for transport to anotherlocation (e.g., final destination), or into a building (e.g., warehouse,business, etc.) or other facility for storage or subsequentdistribution.

FIG. 2 is a system block diagram illustration of the mobile device 20.The mobile device 20 includes device electronics and/or mechanismsoperable to (e.g., fully and/or semi-) autonomously perform one or moretasks such as, for example, loading article(s) onto support surface(s)as illustrated in FIG. 1. The device electronics and/or mechanisms mayinclude a sensor system 44, a device interface system 46, memory 48, amobile platform 50 (e.g., drive system), a manipulator system 52, and acontroller 54. The present invention of course is not to be limited tothe aforesaid electronics and/or mechanisms, and may include alternateelectronics and/or mechanisms other than those specifically describedherein as well as known or future equivalents thereof.

The sensor system 44 is configured to survey the mobile device'ssurroundings and operating environment, and/or to receive location data.The sensor system 44 may include, for example, one or more locationand/or proximity sensors that spatially locate (e.g., triangulate) themobile device 20 relative to, for example, its surrounding environment,its geographic location, and/or one or more locators (e.g., RF tags,physical landmarks, etc.). Examples of location and/or proximity sensorsinclude, but are not limited to, global positioning system (GPS)receivers, radar systems, sonar systems, infrared sensors, lasers,cameras, radio transceivers, etc. The sensor system 44 may additionallyor alternatively include one or more cameras, one or moreelectro-acoustic transducers (e.g., microphones), etc.

The device interface system 46 is configured to receive instructions(e.g., vectors, commands, etc.) in the form of control data from anoperator and/or from a remote control system. The device interfacesystem 46 may also be configured to communicate data to the operatorand/or to the remote control system. The device interface system 46 mayinclude a communication device and/or an operator interface. Thecommunication device may include a cellular, satellite and/or radiotransmitter, and/or a cellular, satellite and/or radio receiver. Theoperator interface may include one or more input and/or output (I/O)devices such as, for example, a display screen, a speaker, a touchscreen, a keypad, a voice command system with an electro-acoustictransducer (e.g., microphone), etc.

The memory 48 (e.g., a non-transitory computer readable medium) may beconfigured to store software (e.g., program instructions) for executionby the controller 54. The memory may include a volatile memory and/or anonvolatile memory. Examples of a volatile memory may include a randomaccess memory (RAM) such as a dynamic random access memory (DRAM), astatic random access memory (SRAM), a synchronous dynamic random accessmemory (SDRAM), a video random access memory (VRAM), etc. Examples of anonvolatile memory may include a read only memory (ROM), an electricallyerasable programmable read-only memory (EEPROM), a computer hard drive,etc.

The mobile platform 50 is configured to move the mobile device 20 tovarious locations within its operative environment. The mobile platform50 may include one or more motorized and/or steerable propulsion devicessuch as, for example, motorized drive wheels, motorized track systems,etc. An embodiment of the mobile platform 50 with motorized drive wheels56 is illustrated in FIG. 3. An embodiment of the mobile platform 50with motorized (e.g., robotic and multi-linkage) track systems 58 isillustrated in FIG. 4. The present invention of course is not limited tothe aforesaid mobile platform configurations and components.

Referring again to FIG. 2, the manipulator system 52 includes one ormore electronically, hydraulically, pneumatically and/or mechanicallyactuated manipulators 60, 62 and 64 configured to move one or morearticles within the operative environment. Examples of a manipulatorinclude, but are not limited to, a robotic manipulator arm (e.g., amulti-axis manipulator arm), a pallet jack, a lift platform, a conveyorsystem, a slide carriage, a crane, etc. In the specific embodimentillustrated in FIG. 3, the manipulator system 52 includes a plurality ofrobotic manipulator arms 60 and 62 and a pallet jack 64. Each of therobotic manipulator aims 60 and 62 includes an end effector 66 connectedto one or more arm members 68 (e.g., linkages). Examples of an endeffector include, but are not limited to, a gripping device, a suctiondevice, an electromagnet, a winch, a clasp, etc. Alternative examples ofa manipulator system are disclosed in U.S. Pat. Nos. 7,902,784;7,719,222 and 7,348,747, each of which is hereby incorporated byreference in its entirety.

Referring again to FIG. 2, the controller 54 may be implemented usinghardware or a combination of hardware and software. The controller 54may include one or more processing devices, analog and/or digitalcircuitry, etc. The controller 54 is in signal communication (e.g.,hardwired or wirelessly connected) with the sensor system 44, the deviceinterface system 46, the memory 48, the mobile platform 50 and themanipulator system 52.

FIG. 5 is a flow diagram of a method for loading a plurality of thearticles (e.g., the articles 22, 24, 26 and 28) on the first supportsurface 30 using the mobile device 20 illustrated in FIGS. 1 and 2. Instep 500, the controller 54 receives control data (e.g., one or morevectors, commands, etc.) through the device interface system 46. Thecontrol data may include inventory data, location data, etc. Theinventory data may, for example, identify the contents, types, states(e.g., amorphous or non-amorphous), masses and/or weights, dimensions,et cetera, of the articles to be arranged on the first support surface.The location data may, for example, identify the respective articlepickup locations, and/or the respective support surface locations.Alternatively, the controller 54 may autonomously determine the articlepickup locations and/or the support surface locations using the sensorsystem 44 where, for example, the articles and/or the support surfacesare configured with RF ID tags, locator beacons, etc. The controller 54may alternatively and/or additionally identify the articles using imageand shape recognition software. The control data may also include othertypes of data such as, for example, scheduling data that designates onwhich support surface(s) and by when and/or in which order the articlesshould be loaded.

In step 502, the controller 54 autonomously models at least some of thearticles (e.g., the articles 22, 24, 26 and 28) in a stable articlearrangement on the first support surface 30. The term “stable” is usedto describe an article arrangement having, for example, (i) a relativelylow center of gravity, (ii) articles that are substantially supported bythe support surface and/or other articles in the arrangement, (iii)articles that are (e.g., equally) balanced on the support surface and/orother articles in the arrangement, etc.

FIG. 6 illustrates an embodiment of a stable article arrangement 70modeled having a geometry that fits and is positioned within a perimeter72 of the first support surface 30. Articles (e.g., base articles 74,free standing articles 76, etc.) contacting the first support surface 30are substantially (e.g., completely) supported by the first supportsurface 30. Articles 78 and 80 stacked on one or more of the basearticles 74 and/or intermediate articles 78 are substantially supportedby the respective articles upon which they are stacked. A bottom surfaceof the intermediate stacked article 78, for example, is substantially(e.g., completely) supported by a top surface of the base article 74,and substantially parallel to the first support surface 30. In anotherexample, each bottom corner of the stacked article 80 is supported bytop surfaces of the base articles 74 and the intermediate stackedarticle 78. The stacked articles 78 and 80 may weigh less than thearticles upon which they are stacked. The stacked articles 78 and 80 mayalso have smaller geometries (e.g., horizontal cross sectional areas,etc) than the articles upon which they are stacked. In some embodiments,the base articles 74 may be staggered relative the stacked articles 80to interlock the articles together with frictional forces. The presentinvention of course is not limited to any particular placement and/orarrangement of the articles.

Referring again to FIGS. 1, 2 and 5, the stable article arrangement maybe computationally modeled using a constrained optimization program oralgorithm such as, for example, a Mix Integer Linear Program (MILP)constrained optimization program. The controller 54, for example, mayinput a plurality of given, fixed and/or variable parameters into aplurality of equality and/or inequality constraint equations, andsubsequently solve the equations using the constrained optimizationprogram. The parameters may be indicative of, for example, (i) a numberof articles to be arranged on the support surface, (ii) dimensions ofthe articles and the support surface, (iii) weights and/or masses of thearticles, (iv) dimensions between each article and one or more otherarticles and/or the support surface, (v) maximum dimensions for thearticle arrangement, etc. The constraint equations may be formulated todescribe, for example, (i) spatial orientations of the articles relativeto other articles and/or the support surface, (ii) relative positions ofthe articles relative to other articles and/or the support surface,(iii) the manner in which the articles are supported, (iv) the order inwhich the articles will be used at their destination, etc. The presentinvention of course is not limited to any particular type ofoptimization programs, parameters and/or constraint equationformulations.

The stable article arrangement may also be modeled to efficiently loadand tightly pack the articles 22, 24, 26 and 28 on the first supportsurface 30. The controller 54, for example, may iteratively change oneor more of the parameter values within the constraint equations to modelthe articles in a plurality of different arrangements. The controller 54may subsequently identify an optimum modeled article arrangement as, forexample, the modeled article arrangement that (i) fits the greatestnumber of articles on the first support surface 30, (ii) has thegreatest arrangement density, and/or (iii) has the greatest controlvolume density.

The term “arrangement density” is used to describe a ratio of acollective volume of the articles in the modeled article arrangement toan overall volume of the modeled article arrangement. The collectivevolume may be determined by adding together volumes of the respectivearticles 22, 24, 26 and 28 in the modeled article arrangement. Theoverall volume may be determined by adding the collective volume tovolumes of respective spaces (e.g., gaps) between the articles 22, 24,26 and 28 in the modeled article arrangement. The controller 54 mayutilize the arrangement density to identify the modeled articlearrangement that packs the articles closest together.

The term “control volume density” is used to describe a ratio of thecollective volume to a control volume. The control volume may be given,or determined from a surface area of the support surface 30, and acontrol volume height (e.g., maximum vertical article arrangementheight). The control volume height may be sized based on, for example, aheight of a shipment container, a trailer, a cargo hold, etc. Thecontrol volume density may be utilized by the controller 54 to identifythe modeled article arrangement that most efficiently packs the articles22, 24, 26 and 28 within the control volume.

In step 504, the controller 54 autonomously schedules the order in whichthe articles 22, 24, 26 and 28 in the optimum modeled articlearrangement are to be gathered and moved to the first support surface 30location. The controller 54, for example, may schedule the base articlesto be gathered and moved before the stacked articles such that thestacked articles may be properly disposed on the base articles. Inanother example, the controller 54 may schedule the article proximate aback side of the first support surface to be gathered and moved beforethe articles proximate to a front side of the first support surface.

In step 506, the mobile device 20 autonomously gathers one or more ofthe articles (e.g., the article 28) in the optimum modeled articlearrangement. The controller 54, for example, may signal the mobileplatform 50 to move the mobile device 20 to the article 28 pickuplocation. The controller 54 may subsequently signal the manipulatorsystem 52 to pick up the article 28 with one or more of the roboticmanipulator arms (e.g., the manipulator arm 60). The end effector 66,for example, may engage (e.g., grip) the article 28 utilizing one of aplurality of (e.g., predetermined) article engagement (e.g., gripping)techniques, and the arm member 68 may lift the article 28 off of therespective source pallet 36. In another example, the controller 54 maysignal the manipulator system 52 to pick up one of the articles with thefirst robotic manipulator arm 60, and to pick up another one of thearticles with the second robotic manipulator arm 62.

The controller 54 may select the article engagement technique used toengage (e.g., grip, push, etc.) the article(s) based on, for example,the type and/or state of the article 28 being gathered, the orientationof the article 28 relative to the source pallet 36 and/or the mobiledevice 20, how the article is to be moved (e.g., lifted, pushed,cradled, etc.), etc. Each of the article engagement techniques maydefine, for example, a specific gripping force or range of grippingforces, a relative end effector spatial orientation, how far to open the(e.g., gripping) end effectors 66, etc. The controller 54, for example,may select a gripping technique that uses a relatively large grippingforce to grasp a heavy article, and a relatively small gripping force tograsp a light or fragile article or an article having an amorphousstate. FIGS. 7 and 8 illustrate a plurality of autonomous mobile robotsutilizing various examples of the gripping techniques.

Referring again to FIGS. 1, 2 and 5, in step 508, the mobile device 20autonomously moves the gathered article(s) from the respective articlepickup location(s) to the first support surface 30 location. Thecontroller 54, for example, may signal the mobile platform 50 to movethe mobile device 20 from the first article pickup location to the firstsupport surface location.

In step 510, the mobile device 20 autonomously arranges the gatheredarticle(s) on the first support surface 30 according to the optimummodeled article arrangement. The controller 54, for example, may signalthe manipulator system 52 to place the article 28 in a particularposition and orientation on the first support surface 30 using, forexample, the robotic manipulator arm 60 holding the article 28.

In step 512, the mobile device 20 may repeat the steps 506, 508 and 510to gather, move and arrange the remaining articles of the optimummodeled article arrangement.

In step 514, the mobile device 20 may autonomously secure the articles22, 24, 26 and 28 together and/or to the first support surface 30. Themobile device 20, for example, may utilize the manipulator system 52 towrap the articles together in packaging material such as, for example,shrink wrap, transparent or non-transparent plastic wrap, etc. Inanother example, the mobile device 20 may utilize the manipulator system52 to secure the article arrangement to the first support surface 30with, for example, tie down straps.

In some embodiments, the step 508 may be omitted from the method of FIG.5 where, for example, the articles 22, 24, 26 and 28 may be respectivelygathered and subsequently (e.g., immediately) arranged on the firstsupport surface 30. Such an embodiment is illustrated in FIGS. 9A-9Fwhere the pallet jack 64 is engaged with a pallet 82 that defines thefirst support surface 30. In other embodiments, the pallet 82 may beutilized to temporarily hold the gathered article(s) during movement ofthe article(s) between the article pickup location(s) and the firstsupport surface location; i.e., where the step 508 is not omitted.

In some embodiments, where the articles identified in the step 500cannot fit within the control volume of a single support surface, thecontroller 54 may model a first grouping of the articles in a firststable article arrangement on the first support surface, a secondgrouping of the articles in a second stable article arrangement on asecond support surface, etc. The controller 54 may also utilize theconstrained optimization program to reduce (e.g., minimize) the numberof support surfaces necessary to hold the identified articles. In suchan embodiment, the mobile device 20 may perform the steps 506, 508 and510 iteratively for each of the modeled article arrangements, orsubstantially concurrently in a manner that reduces (e.g., minimizes)loading time. The loading time may be minimized, for example, bygathering and arranging a similar type of the articles onto a pluralityof the support surfaces before beginning the gathering and arranging ofanother type of the articles. Alternatively, the mobile device 20 maysignal a second mobile device to load the second support surface whilethe mobile device 20 loads the first support surface, etc.

In some embodiments, the constraint equations used in the step 502 maybe weighted in order to arrange certain types of the articles at certainpositions within the modeled article arrangement. Such weighting may beuseful, for example, where certain types of the articles are scheduledto be unpacked at a final destination before other types of thearticles, are scheduled to be removed mid-transit, etc.

In alternate embodiments, a remote control system may be provided thatautonomously controls one or more mobile devices to load one or moresupport surfaces. The control system, for example, may perform the steps500, 502 and 504 and subsequently autonomously coordinate and signal afirst of the mobile devices and a second of the mobile devices to gatherand arrange respective articles on the first support surface. In anotherexample, the control system may autonomously coordinate and signal thefirst of the mobile devices and the second of the mobile devices togather and arrange respective articles on different support surfacesthat are to be transported to and/or stored in, for example, the samelocation. In some embodiments, the control system may select which ofthe mobile devices loads which of the support surfaces based on, forexample, the mobile device's proximity to the article pickup locationsand/or support surface locations, the mobile device's load capacities orfuel/battery level, etc.

During transportation, warehousing and/or storage of a loaded supportsurface, one or more articles in an article arrangement 84 on thesupport surface may be damaged, stolen, replaced with other articles, orotherwise tampered with, which may alter the geometry and appearance ofthe article arrangement as illustrated in FIGS. 10A and 10B. Thegeometry and appearance of the article arrangement 84 may change, forexample, where (i) one of the perimeter articles is missing, (ii) one ofthe articles is replaced with an article having a different size orphysical appearance, (iii) the packaging material wrapped around thearticle arrangement is removed and replaced with different packagingmaterial or wrapped in a different manner, etc.

FIG. 11 illustrates a system 86 for monitoring articles in an articlearrangement on a loaded support surface. The system 86 may include themobile device 20 and an article inspection system 88. The articleinspection system 88 includes a processing device 90 that is in signalcommunication with a communication device 92 as well as a camera 94, aprinter 96 and/or a display screen 98. The mobile device 20 may belocated at a first location (e.g., an article distribution/loadingfacility, etc.), and the article inspection system 88 may be located ata second location (e.g., an article receiving facility, another articledistribution/loading facility, a final destination, etc.).

FIG. 12 is a flow diagram of a method for monitoring the articles in anarticle arrangement using the system 86 of FIG. 11. In step 1200, themobile device 20 performs the method of FIG. 5 to arrange the articleson the support surface according to an autonomously modeled articlearrangement.

In step 1202, the controller 54 (see FIG. 2) generates image data fromthe modeled article arrangement, where the image data is indicative ofan image of the arranged articles on the loaded support surface. Theimaged data may be generated, for example, by processing the modeledarticle arrangement with given or determined dimensions and geometriesof the articles in the modeled article arrangement. The controller 54may also fit a surface to the modeled article arrangement to representthe packaging material wrapped around the articles in the step 514 (see,e.g., FIGS. 10A and 10B). It is contemplated that image data may includea datum that establishes an image coordinate system for the image.

In step 1204, the mobile device 20 transmits the image data to theprocessing device 90 through the communication device 92. The image datamay be transmitted before, after and/or during the transportation of theloaded support surface from the first location to the second location.

In step 1206, the image data received by the processing device 90 isutilized to inspect the arranged articles on the loaded support surfaceat the second location. The processing device 90, for example, maysignal the printer 96 to print the image of the arranged articles. Aninspector, security guard, dock worker, government employee, etc. maysubsequently visually compare the arranged articles on the loadedsupport surface to the printed image to determine whether any of thearticles are damaged, tampered with, missing, etc. In another example,the processing device 90 may signal the display screen 98 to display theimage of the arranged articles such that it may be visually compared tothe arranged articles on the loaded support surface.

In other embodiments, the processing device 90 may receive second imagedata from the camera 94, where the second image data is indicative of asecond image of the arranged articles on the loaded support surface atthe second location. The processing device 90 may subsequently compareone or more image characteristics between the first image data and thesecond image data to determine whether any of the articles are damaged,tampered with, missing, etc. The image characteristics may be comparedutilizing, for example, image and/or shape recognition software. In someembodiments, the inspection system 88 may autonomously generate thesecond image data with the camera 94 and compare the first and secondimage data where, for example, the inspection system 88 is configured asthe mobile device 20 described above.

In other embodiments, the method of FIG. 12 may be utilized to inspectthe articles on the loaded support surface before they have beentransported to the second location, or during transit to the secondlocation.

A person of skill in the art will recognize that the methods of FIGS. 5and 12 and the various embodiments thereof may be performed by devicesand systems other than those specifically described above. A stationaryrobotic device, a mobile or stationary crane system, a conveyor systemas well as various stationary or mobile manipulator devices, forexample, may be adapted to perform one or more of the afore-describedmethods.

A person of skill in the art will also recognize that the modeledarticle arrangement utilized in the method of FIG. 12 need not beautonomously modeled as set forth in the step 1200. The modeled articlearrangement, for example, maybe developed by a worker to provide a planof how the articles should be arranged by loading personnel on thesupport surface.

It is to be understood that the terminology used herein is used for thepurpose of describing specific embodiments, and is not intended to limitthe scope of the present invention. It should be noted that as usedherein, the singular forms of “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. In addition,unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs.

Although the description above contains many specific examples, theseshould not be construed as limiting the scope of the embodiments of thepresent disclosure, but as merely providing illustrations of some of thepresently preferred embodiments of the present invention. It will beappreciated by those skilled in the art that changes could be made tothe embodiments described above without departing from the broadinventive concept thereof. It is to be understood therefore that thisdisclosure is not limited to the specific embodiments disclosed herein,but it is intended to cover modifications within the spirit and scope ofthe embodiments of the present disclosure. Accordingly, the presentinvention is not to be restricted except in light of the attached claimsand their equivalents.

What is claimed is:
 1. A method for autonomously determining potentialtampering or damage of a shipment of a plurality of articles at adestination point without any human intervention, the method comprising:providing a list of the plurality of articles to be shipped to adestination point, the list of plurality of articles comprising a firstarticle having a rectangular shape and a second article having anon-rectangular shape and a third article having an amorphous shape, anda fourth article having a shape wherein the shape of the fourth articleis rectangular, non-rectangular or amorphous; modeling the plurality ofarticles provided on the list in an article arrangement for placement ona first support surface using a first processing device to yield amodeled article arrangement; arranging the plurality of articles on thefirst support surface according to the modeled article arrangement usinga fully autonomous mobile robot to yield a first article arrangement;electronically generating a first set of digital image data from themodeled article arrangement, wherein the digital image data isindicative of a digital image of all of the first article arrangement onthe first support surface at a first point in time at a first shippingpoint for the plurality of articles; providing a second set of digitalimage data from a digital camera, the second set of digital image databeing indicative of a second article arrangement on a second supportsurface at a second point in time at a second destination point for thesecond article arrangement, the second destination point beinggeographically remote from the first shipping point, wherein the firstarticle arrangement on the first support surface are in a differentconfiguration from the second article arrangement on the second supportsurface; receiving the second set of digital image data by a secondprocessing device; and comparing autonomously and electronically, viathe second processing device, the first set of digital image datagenerated from the modeled article arrangement at the first point intime at the first shipping point with the second set of digital imagedata of the second article arrangement at the second point in time atthe destination point without any human intervention needed to comparethe first set of digital image data and the second set of digital imagedate; identifying a set of differences between the first set of digitalimage data and the second set of digital image data, the set ofdifferences comprising at least one or more of a missing article, areoriented article or damage to one or more articles to yield anidentified set of differences between the first set of digital imagedata and the second set of digital image data; and determining potentialtampering or damage of the first article arrangement on the firstsupport surface by a fully autonomous mobile robot classifying theidentified set of differences between the first set of digital imagedata and the second set of digital image data wherein the classificationis selected from the group consisting of potential tampering or damage.2. The method as defined in claim 1, wherein the second processingdevice utilizes image and shape recognition software to compare thefirst set of digital image data and the second set of digital imagedata.
 3. The method as defined in claim 1, wherein the modeled articlearrangement includes a particular predetermined control volume, theparticular predetermined control volume being variable depending upon atype of support surface being utilized both with and without pallets. 4.The method as defined in claim 3, wherein at least one of the firstsupport surface or the second support surface is a floor of a shippingcontainer and the particular predetermined control volume is adjusted toprovide maximum efficiency in modeling the first article arrangement andthe second article arrangement on the floor of the shipping containerboth with and without pallets.
 5. The method as defined in claim 3,wherein at least one of the first support surface or the second supportsurface is a floor of a trailer in use with a tractor trailer truck andthe particular predetermined control volume is adjusted to providemaximum efficiency in modeling the first article arrangement and thesecond article arrangement on the floor of the trailer both with andwithout pallets.
 6. The method as defined in claim 1, wherein similararticles are loaded onto the first support surface prior to dissimilaritems while still achieving a stable modeling of articles.
 7. The methodas defined in claim 1, including encasing the first article arrangementand the second article arrangement in shrink-wrap and autonomouslydetermining without any human intervention if the shrink-wrap has beentampered with.
 8. The method as defined in claim 1, wherein both thefirst set of digital image data and the second set of digital image datafurther identify a particular type of each of the articles arranged onthe first support surface with a particular article description oridentifier.